{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Understanding RNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What we use till now\n",
    "Neural network that we use till now like dense, convnets have no memory.\n",
    "\n",
    "They were independent and no state were kept it between there input.\n",
    "\n",
    "With such network, to process sequence or series data we have to show entire data at once like we did with imdb.\n",
    "\n",
    "An entire movie review was transformed into a single large vector and processed in one go. Such networks are called\n",
    "feedforward networks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Recurrent neural network\n",
    "\n",
    "A recurrent neural network (RNN) processes sequences and keep them in memmory\n",
    "\n",
    "It works in batches\n",
    "\n",
    "We use it on sequentioal data where future is dependent on past\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "# RNN Example : Pseudocode RNN\n",
    "\n",
    "state_t = 0                          # The state at t\n",
    "for input_t in input_sequence:       # Iterates over sequence elements\n",
    "    output_t = f(input_t, state_t)   \n",
    "    state_t = output_t               # The previous output becomes the state for the next iteration.\n",
    "\n",
    "# Another RNN Example : More detailed pseudocode for the RNN\n",
    "\n",
    "state_t = 0\n",
    "for input_t in input_sequence:\n",
    "    output_t = activation(dot(W, input_t) + dot(U, state_t) + b)\n",
    "    state_t = output_t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SimpleRNN\n",
    "\n",
    "SimpleRNN takes inputs of shape (batch_size, timesteps, input_features), rather than (timesteps,input_features)\n",
    "\n",
    "SimpleRNN can be run in two different modes:\n",
    "1. (a 3D tensor of shape (batch_size, timesteps, output_features))\n",
    "2. (a 2D tensor of shape (batch_size, output_features))\n",
    "\n",
    "These two modes are controlled by the return_sequences constructor argument"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding (Embedding)        (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn (SimpleRNN)       (None, 32)                2080      \n",
      "=================================================================\n",
      "Total params: 322,080\n",
      "Trainable params: 322,080\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# example that uses SimpleRNN and returns only the output at the last timestep\n",
    "# (a 2D tensor of shape (batch_size, output_features))\n",
    "\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Embedding, SimpleRNN\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn_1 (SimpleRNN)     (None, None, 32)          2080      \n",
      "=================================================================\n",
      "Total params: 322,080\n",
      "Trainable params: 322,080\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# The following example returns the full state sequence:\n",
    "# (a 3D tensor of shape (batch_size, timesteps, output_features))\n",
    "\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Embedding, SimpleRNN\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It’s sometimes useful to stack several recurrent layers one after the other in order to\n",
    "increase the representational power of a network. In such a setup, you have to get all\n",
    "of the intermediate layers to return full sequence of outputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_2\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_2 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn_2 (SimpleRNN)     (None, None, 32)          2080      \n",
      "_________________________________________________________________\n",
      "simple_rnn_3 (SimpleRNN)     (None, None, 32)          2080      \n",
      "_________________________________________________________________\n",
      "simple_rnn_4 (SimpleRNN)     (None, None, 32)          2080      \n",
      "_________________________________________________________________\n",
      "simple_rnn_5 (SimpleRNN)     (None, 32)                2080      \n",
      "=================================================================\n",
      "Total params: 328,320\n",
      "Trainable params: 328,320\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.add(SimpleRNN(32))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Rnn on IMDB Movie Review Classification Problem\n",
    "Enough with the examples lets implement the rnn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25000 train sequences\n",
      "25000 test sequences\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.datasets import imdb  # loading IMdb from keras\n",
    "max_features = 10000                        # Max word\n",
    "\n",
    "(input_train, y_train), (input_test, y_test) = imdb.load_data(num_words=max_features)\n",
    "\n",
    "print(len(input_train), 'train sequences')\n",
    "print(len(input_test), 'test sequences')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input_train shape: (25000, 500)\n",
      "input_test shape: (25000, 500)\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.preprocessing import sequence\n",
    "maxlen = 500                # lenth of sentence\n",
    "\n",
    "input_train = sequence.pad_sequences(input_train, maxlen=maxlen)\n",
    "input_test = sequence.pad_sequences(input_test, maxlen=maxlen)\n",
    "\n",
    "print('input_train shape:', input_train.shape)\n",
    "print('input_test shape:', input_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training the model with Embedding and SimpleRNN layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 60s 3ms/sample - loss: 0.6331 - acc: 0.6238 - val_loss: 0.4879 - val_acc: 0.7742\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 58s 3ms/sample - loss: 0.3861 - acc: 0.8366 - val_loss: 0.3921 - val_acc: 0.8236\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 58s 3ms/sample - loss: 0.2876 - acc: 0.8843 - val_loss: 0.4746 - val_acc: 0.7700\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 59s 3ms/sample - loss: 0.2240 - acc: 0.9156 - val_loss: 0.3597 - val_acc: 0.8598\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 59s 3ms/sample - loss: 0.1744 - acc: 0.9358 - val_loss: 0.4056 - val_acc: 0.8252\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 58s 3ms/sample - loss: 0.1186 - acc: 0.9593 - val_loss: 0.5228 - val_acc: 0.7796\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 59s 3ms/sample - loss: 0.0818 - acc: 0.9732 - val_loss: 0.4432 - val_acc: 0.8640\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 59s 3ms/sample - loss: 0.1627 - acc: 0.9450 - val_loss: 0.5331 - val_acc: 0.8242\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 58s 3ms/sample - loss: 0.1165 - acc: 0.9661 - val_loss: 0.4842 - val_acc: 0.8530\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 59s 3ms/sample - loss: 0.0366 - acc: 0.9893 - val_loss: 0.5350 - val_acc: 0.8340\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Embedding, SimpleRNN\n",
    "from tensorflow.keras.layers import Dense\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(max_features, 32))     # max_features = 10000\n",
    "model.add(SimpleRNN(32))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])\n",
    "\n",
    "history = model.fit(input_train, y_train,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Plotting results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXwUVbbA8d8hhH0LixvIpiiyBWIEHVBREcENRUdBUFxRFBeUmUFxxIeiPnUc13HguQ8Iw8i4K4qK4i5BCQrIIiBGEMIWVoGQ8/64ldAJnaSTdHd1Kuf7+fQn3bWeru6cun3vrVuiqhhjjAmuan4HYIwxJrYs0RtjTMBZojfGmICzRG+MMQFnid4YYwLOEr0xxgScJfoqSESSRGS7iLSM5rJ+EpEjRSTqfYVFpI+IrAp5vUREToxk2XLs6xkRuaO86xtTnOp+B2BKJyLbQ17WAXYD+7zX16rqlLJsT1X3AfWivWxVoKpHR2M7InI1MFRVe4ds++pobNuYoizRVwKqWpBovRLj1ar6QXHLi0h1Vc2NR2zGlMa+j/6zqpsAEJF7ReTfIjJVRLYBQ0XkBBH5SkS2iMhaEXlcRJK95auLiIpIa+/1ZG/+uyKyTUS+FJE2ZV3Wm99fRJaKSI6IPCEin4vI5cXEHUmM14rIchHZLCKPh6ybJCJ/F5GNIvIT0K+E43OniEwrMu0pEXnEe361iCz23s9PXmm7uG1liUhv73kdEfmXF9tC4Ngw+13hbXehiJzrTe8MPAmc6FWLbQg5tneHrH+d9943ishrInJoJMemLMc5Px4R+UBENonIbyLy55D9/NU7JltFJENEDgtXTSYin+V/zt7xnOPtZxNwp4i0E5HZ3nvZ4B23hiHrt/LeY7Y3/zERqeXFfEzIcoeKyE4RaVLc+zVhqKo9KtEDWAX0KTLtXmAPcA7u5F0bOA7ogfvV1hZYCoz0lq8OKNDaez0Z2ACkA8nAv4HJ5Vj2IGAbMMCbdyuwF7i8mPcSSYyvAw2B1sCm/PcOjAQWAi2AJsAc93UOu5+2wHagbsi21wPp3utzvGUEOBXYBXTx5vUBVoVsKwvo7T1/GPgYSAFaAYuKLHsRcKj3mVzixXCwN+9q4OMicU4G7vae9/Vi7ArUAv4BfBTJsSnjcW4IrANuBmoCDYDu3rzbgUygnfceugKNgSOLHmvgs/zP2XtvucAIIAn3fTwKOA2o4X1PPgceDnk/P3jHs663fE9v3iRgQsh+bgNe9fv/sLI9fA/AHmX8wIpP9B+Vst5o4D/e83DJ+58hy54L/FCOZa8EPg2ZJ8Baikn0EcZ4fMj8/wKjvedzcFVY+fPOLJp8imz7K+AS73l/YGkJy74F3OA9LynRrw79LIDrQ5cNs90fgLO856Ul+heB+0LmNcC1y7Qo7diU8ThfCmQUs9xP+fEWmR5Jol9RSgwXAnO95ycCvwFJYZbrCawExHs9HxgY7f+roD+s6iY4fgl9ISLtReRt76f4VmA80LSE9X8Leb6Tkhtgi1v2sNA41P1nZhW3kQhjjGhfwM8lxAvwMjDYe34JUNCALSJni8jXXtXFFlxpuqRjle/QkmIQkctFJNOrftgCtI9wu+DeX8H2VHUrsBloHrJMRJ9ZKcf5cGB5MTEcjkv25VH0+3iIiEwXkV+9GF4oEsMqdQ3/hajq57hfB71EpBPQEni7nDFVWZbog6No18KJuBLkkaraALgLV8KOpbW4EicAIiIUTkxFVSTGtbgEka+07p//BvqISAtc1dLLXoy1gVeA+3HVKo2A9yOM47fiYhCRtsDTuOqLJt52fwzZbmldQdfgqoPyt1cfV0X0awRxFVXScf4FOKKY9Yqbt8OLqU7ItEOKLFP0/f0vrrdYZy+Gy4vE0EpEkoqJ4yVgKO7Xx3RV3V3McqYYluiDqz6QA+zwGrOujcM+3wLSROQcEamOq/dtFqMYpwO3iEhzr2HuLyUtrKrrcNULzwNLVHWZN6smrt44G9gnImfj6pIjjeEOEWkk7jqDkSHz6uGSXTbunHc1rkSfbx3QIrRRtIipwFUi0kVEauJORJ+qarG/kEpQ0nF+A2gpIiNFpIaINBCR7t68Z4B7ReQIcbqKSGPcCe43XKN/kogMJ+SkVEIMO4AcETkcV32U70tgI3CfuAbu2iLSM2T+v3BVPZfgkr4pI0v0wXUbMAzXODoRV6KNKS+ZXgw8gvvHPQL4DleSi3aMTwMfAt8Dc3Gl8tK8jKtzfzkk5i3AKOBVXIPmhbgTViTG4X5ZrALeJSQJqeoC4HHgG2+Z9sDXIevOApYB60QktAomf/2ZuCqWV731WwJDIoyrqGKPs6rmAKcDF+Aaf5cCJ3uzHwJewx3nrbiG0Vpeldw1wB24hvkji7y3cMYB3XEnnDeAGSEx5AJnA8fgSvercZ9D/vxVuM95j6p+Ucb3btjfwGFM1Hk/xdcAF6rqp37HYyovEXkJ18B7t9+xVEZ2wZSJKhHph/sp/juue14urlRrTLl47R0DgM5+x1JZWdWNibZewArcT/p+wHnWeGbKS0Tux/Xlv09VV/sdT2VlVTfGGBNwVqI3xpiAS7g6+qZNm2rr1q39DsMYYyqVefPmbVDVsN2ZS030IvIcruvTelXtFGa+AI/hLkHfibsM+ltv3jDgTm/Re1X1xdL217p1azIyMkpbzBhjTAgRKfbq8Eiqbl6ghJEBceOGtPMew3H9m/EurBiHG0ypOzBORFIiC9kYY0y0lJroVXUO7kKS4gwAXlLnK6CRN5zqGcAsVd2kqptxF4iUdMIwxhgTA9FojG1O4QGMsrxpxU0/gIgM98a6zsjOzo5CSMYYY/JFozE23OBPWsL0AyeqTsJdXk16evoBy+zdu5esrCx+//33isRpYqxWrVq0aNGC5OTihm8xxvghGok+i8Ij+LXAXfaeBfQuMv3jcu0gK4v69evTunVrXNuvSTSqysaNG8nKyqJNmzalr2CMiZtoVN28AVzmjW53PJCjqmuB94C+IpLiNcL29aaV2e+//06TJk0syScwEaFJkyb2q8uYcpgyBVq3hmrV3N8pU0pbo2wi6V45FVcybyoiWbieNMkAqvpP4B1c18rluO6VV3jzNonIPbiRBQHGq2pJjbqlxVHeVU2c2GdkTNlNmQLDh8POne71zz+71wBDyjteaRGlJnpVHVzKfAVuKGbec8Bz5QvNGGOCb+zY/Uk+386dbnq0Er0NgRCBjRs30rVrV7p27cohhxxC8+bNC17v2bMnom1cccUVLFmypMRlnnrqKaZE+zebMSahrS5mqLbippdHwg2BEA1Tpriz4erV0LIlTJhQsTNjkyZNmD9/PgB333039erVY/To0YWWKbgJb7Xw587nn3++1P3ccEPYH0bGmABr2dJV14SbHi2BK9Hn13f9/DOo7q/vikVBefny5XTq1InrrruOtLQ01q5dy/Dhw0lPT6djx46MHz++YNlevXoxf/58cnNzadSoEWPGjCE1NZUTTjiB9evXA3DnnXfy6KOPFiw/ZswYunfvztFHH80XX7gb6+zYsYMLLriA1NRUBg8eTHp6esFJKNS4ceM47rjjCuLLH6V06dKlnHrqqaSmppKWlsaqVasAuO++++jcuTOpqamMHTs2+gfLGBPWhAlQp07haXXquOnRErhEX1J9VywsWrSIq666iu+++47mzZvzwAMPkJGRQWZmJrNmzWLRokUHrJOTk8PJJ59MZmYmJ5xwAs89F74ZQ1X55ptveOihhwpOGk888QSHHHIImZmZjBkzhu+++y7sujfffDNz587l+++/Jycnh5kzZwIwePBgRo0aRWZmJl988QUHHXQQb775Ju+++y7ffPMNmZmZ3HbbbVE6OsaY0gwZApMmQatWIOL+TpoUvfp5CGCij0d9V6gjjjiC4447ruD11KlTSUtLIy0tjcWLF4dN9LVr16Z///4AHHvssQWl6qIGDhx4wDKfffYZgwYNAiA1NZWOHTuGXffDDz+ke/fupKam8sknn7Bw4UI2b97Mhg0bOOeccwB3gVOdOnX44IMPuPLKK6lduzYAjRs3LvuBMMaU25AhsGoV5OW5v9FM8hDAOvp41HeFqlu3bsHzZcuW8dhjj/HNN9/QqFEjhg4dGrZfeY0aNQqeJyUlkZubG3bbNWvWPGCZSG4Us3PnTkaOHMm3335L8+bNufPOOwviCNcFUlWta6QxARa4En086ruKs3XrVurXr0+DBg1Yu3Yt771XruvDStSrVy+mT58OwPfffx/2F8OuXbuoVq0aTZs2Zdu2bcyYMQOAlJQUmjZtyptvvgm4C9F27txJ3759efbZZ9m1axcAmzaV+3IHUwnE+uIck3gCV6LP/8kTzV43kUpLS6NDhw506tSJtm3b0rNnz6jv48Ybb+Syyy6jS5cupKWl0alTJxo2bFhomSZNmjBs2DA6depEq1at6NGjR8G8KVOmcO211zJ27Fhq1KjBjBkzOPvss8nMzCQ9PZ3k5GTOOecc7rnnnqjHbvwXj4tzTOJJuHvGpqena9EbjyxevJhjjjnGp4gSS25uLrm5udSqVYtly5bRt29fli1bRvXqiXHOts8qsbVuHb5qs1UrVzdsKi8Rmaeq6eHmJUZ2MBHbvn07p512Grm5uagqEydOTJgkbxJfvDsrmMQQuDr6oGvUqBHz5s0jMzOTBQsW0LdvX79DMpVIcZ0SYtVZIdFVlfYKS/TGVCF+dlZINPG8uNJvluiNqULicXFOZRHviyv9ZInemDhJlGqCWF+cU1lUpfYKS/TGxEFVqiaoLKpSe4Ul+gj07t37gIufHn30Ua6//voS16tXrx4Aa9as4cILLyx220W7kxb16KOPsjPkN+aZZ57Jli1bIgndJIiqVE1QWVSl9gpL9BEYPHgw06ZNKzRt2rRpDB5c4j1ZChx22GG88sor5d5/0UT/zjvv0KhRo3Jvz8RfVaomqCyqUnuFJfoIXHjhhbz11lvs3r0bgFWrVrFmzRp69epV0K89LS2Nzp078/rrrx+w/qpVq+jUqRPghicYNGgQXbp04eKLLy4YdgBgxIgRBUMcjxs3DoDHH3+cNWvWcMopp3DKKacA0Lp1azZs2ADAI488QqdOnejUqVPBEMerVq3imGOO4ZprrqFjx4707du30H7yvfnmm/To0YNu3brRp08f1q1bB7i++ldccQWdO3emS5cuBUMozJw5k7S0NFJTUznttNOicmyriqpUTVCZVJX2ioiutBGRfsBjQBLwjKo+UGR+K9wtA5sBm4ChqprlzdsHfO8tulpVz61IwLfcAmGGX6+Qrl3By5FhNWnShO7duzNz5kwGDBjAtGnTuPjiixERatWqxauvvkqDBg3YsGEDxx9/POeee26xg4Q9/fTT1KlThwULFrBgwQLS0tIK5k2YMIHGjRuzb98+TjvtNBYsWMBNN93EI488wuzZs2natGmhbc2bN4/nn3+er7/+GlWlR48enHzyyaSkpLBs2TKmTp3K//3f/3HRRRcxY8YMhg4dWmj9Xr168dVXXyEiPPPMMzz44IP87W9/45577qFhw4Z8/7372DZv3kx2djbXXHMNc+bMoU2bNjYeThlNmFB46AEIbjWBSTylluhFJAl4CugPdAAGi0iHIos9DLykql2A8cD9IfN2qWpX71GhJO+n0Oqb0GobVeWOO+6gS5cu9OnTh19//bWgZBzOnDlzChJuly5d6NKlS8G86dOnk5aWRrdu3Vi4cGHYActCffbZZ5x//vnUrVuXevXqMXDgQD799FMA2rRpQ9euXYHih0LOysrijDPOoHPnzjz00EMsXLgQgA8++KDQ3a5SUlL46quvOOmkk2jTpg1gQxmXVVWqJjCJJ5ISfXdguaquABCRacAAIDQLdQBGec9nA69FM8hQJZW8Y+m8887j1ltv5dtvv2XXrl0FJfEpU6aQnZ3NvHnzSE5OpnXr1mGHJg4VrrS/cuVKHn74YebOnUtKSgqXX355qdspaZyi/CGOwQ1zHK7q5sYbb+TWW2/l3HPP5eOPP+buu+8u2G7RGG0o44obMsQSu/FHJHX0zYFfQl5nedNCZQIXeM/PB+qLSBPvdS0RyRCRr0TkvHA7EJHh3jIZ2dnZZQg/furVq0fv3r258sorCzXC5uTkcNBBB5GcnMzs2bP5OdyIUSFOOumkghuA//DDDyxYsABwQxzXrVuXhg0bsm7dOt59992CderXr8+2bdvCbuu1115j586d7Nixg1dffZUTTzwx4veUk5ND8+buo3zxxRcLpvft25cnn3yy4PXmzZs54YQT+OSTT1i5ciVgQxmbikmUawqqikgSfbhiXNGi5GjgZBH5DjgZ+BXIv5tGS29EtUuAR0XkiAM2pjpJVdNVNb1Zs2aRRx9ngwcPJjMzs+AOTwBDhgwhIyOD9PR0pkyZQvv27UvcxogRI9i+fTtdunThwQcfpHv37oC7W1S3bt3o2LEjV155ZaEhjocPH07//v0LGmPzpaWlcfnll9O9e3d69OjB1VdfTbdu3SJ+P3fffTd//OMfOfHEEwvV/995551s3ryZTp06kZqayuzZs2nWrBmTJk1i4MCBpKamcvHFF0e8H2NC2TUF8VfqMMUicgJwt6qe4b2+HUBV7y9m+XrAj6raIsy8F4C3VLXYvoY2THHlZp+VKY0NlRwbJQ1THEmJfi7QTkTaiEgNYBDwRpEdNBWR/G3djuuBg4ikiEjN/GWAnhSu2zfGVDF2TUH8lZroVTUXGAm8BywGpqvqQhEZLyL5vWh6A0tEZClwMJDfaewYIENEMnGNtA+oqiV6Y6owu6Yg/iK6YEpV31HVo1T1CFWd4E27S1Xf8J6/oqrtvGWuVtXd3vQvVLWzqqZ6f58tb6CJdicsc6BE/Yys4S+xVKWhBxJFpbgytlatWmzcuDFhE4lxSX7jxo3UqlXL71AKsYa/xGPXFMRfpbhn7N69e8nKyiq1X7nxV61atWjRogXJycl+h1LAGv5MVVHp7xmbnJxccEWmMWVhDX/GVJKqG2PKyxr+jLFEbwLOGv6MsURvAs4a/oypJHX0xlSEDSZmqjor0RtjTMBZojfGmICzRG+MMQFnid7EjA09YExisMZYExP5Qw/k3yM1f+gBsIZRY+LNSvQmJsaOLXwjbHCvx471Jx5jqjJL9CYmbOgBYxKHJXoTEzb0gDGJwxK9iQkbesCYxGGJ3sSEDT1gTOKwXjcmZmzoAWMSQ0QlehHpJyJLRGS5iIwJM7+ViHwoIgtE5GMRaREyb5iILPMew6IZvDHGmNKVmuhFJAl4CugPdAAGi0iHIos9DLykql2A8cD93rqNgXFAD6A7ME5EUqIXvjHGmNJEUqLvDixX1RWqugeYBgwoskwH4EPv+eyQ+WcAs1R1k6puBmYB/SoetjHGmEhFkuibA7+EvM7ypoXKBC7wnp8P1BeRJhGua4wxJoYiSfQSZlrRO4qPBk4Wke+Ak4FfgdwI10VEhotIhohkZGdnRxCSMcaYSEWS6LOAw0NetwDWhC6gqmtUdaCqdgPGetNyIlnXW3aSqqaranqzZs3K+BaMMcaUJJJEPxdoJyJtRKQGMAh4I3QBEWkqIvnbuh14znv+HtBXRFK8Rti+3jRjjDFxUmqiV9VcYCQuQS8GpqvqQhEZLyLneov1BpaIyFLgYGCCt+4m4B7cyWIuMN6bZowxJk5E9YAqc1+lp6drRkaG32EYY0ylIiLzVDU93DwbAiGA7IYfxphQNgRCwNgNP4wxRVmJPmDshh/GmKIs0QeM3fDDGFOUJfqAsRt+GGOKskQfMHbDD2NMUZboA8Zu+GGMKcp63QSQ3fDDGBPKSvTGGBNwluiNMSbgLNEbY0zAWaI3xpiAs0RvjDEBZ4nemCpm3ToYNAhee83vSEy8WKI3pgrJy4NLL4V//xvOPx8uuADWHHDPNxM0luhNTH3/vUsq+/b5HYkB+N//hVmz4Kmn4P774Z13oEMHmDjRnQRMMFmiNzGzaRP06+eqCbp2hZkzIcHuc1OlfP45/PWv7vMYMQLGjHEn4mOPheuug9694ccf/Y7SxIIlehMzN9wA69fDQw+5oZL794czzoDMTL8jq3o2boTBg92NaCZOdMNjABx5JHzwATz3HPzwA6SmwvjxsGePr+GaKIso0YtIPxFZIiLLRWRMmPktRWS2iHwnIgtE5ExvemsR2SUi873HP6P9BkximjoVpk2D//kfGD0aFi+GRx+FefOgWze44grIyvI7yqpB1R3v335z1WgNGhSeL+LmL14MAwfCuHHuM/riC3/iNTGgqiU+gCTgJ6AtUAPIBDoUWWYSMMJ73gFY5T1vDfxQ2j5CH8cee6yaym31atWGDVX/8AfVvXsLz9u0SXX0aNUaNVRr11a9807VrVv9ibOq+PvfVUH1scciW/7tt1VbtlQVUb3+etWcnNjG56eff1YdNUq1b1/Vhx5S/fFHvyMqPyBDi8vjxc3Q/Un8BOC9kNe3A7cXWWYi8JeQ5b9QS/RV0r59qqeeqlqvnury5cUvt2KF6uDB7ht40EGqTz994EnBVNzcuarJyaoDBqjm5UW+3rZtqjff7JJ98+aqr70Wuxj9kJmpOnSoalKSavXqqu3bu+8iqB51lCuMzJlTub6TFU30FwLPhLy+FHiyyDKHAt8DWcBm4Fjdn+h3AN8BnwAnlrY/S/SVW37p8ZlnIlv+669VTzzRrdO+veqbb5YtIZnibdmi2rat6uGHq27cWL5tfP21aufO7vO54ALVNWuiG2M85eWpfvSRar9+7v3UretK8z//7OavWqX65JOudJ+c7JZp3Fj10ktV//OfxP/lWdFE/8cwif6JIsvcCtym+0v0i3D1/zWBJt70Y4FfgAZh9jEcyAAyWrZsGbcDY6Lrhx9Ua9ZUPffcsiXrvDxXYjzqKPeN7N1bNSMjdnFWBXl5qhdd5Eqsn39esW3t2aN6333us23YUHXiRPfLrbLIzVWdPl01PX3/L8gJE1w1YnFyctw6Q4e6ZA+uuvGMM1SfespVTyaaeFTdLAQOD3m9AjgozLY+BtJL2p+V6Cun3btVu3ZVbdZMdd268m1jzx5Xomra1H0zhw7dX9oyZfPPf7pjeP/90dvm0qWqp5zitnviiaqLF0dv27GwY4dLym3bupjbtXMnqV27yradvXtVP/lE9bbbVI88cn8VT9euqnfd5QolifArtKKJvrqXuNuENMZ2LLLMu8Dl3vNjgDWAAM2AJG96W+BXoHFJ+7NEXzmNGeO+Ta+/XvFtbdmievvtqrVquVLkX/7ippnIZGa6Y9e3b/RL3nl5qs89p5qS4kq4//M/7iSfSDZscHHlFxh69FCdMcOV7CsqL8+d4B58ULVXL9Vq1dw+DjtM9dprXUN2WU8k0VKhRO/W50xgqdf7Zqw3bTxwrve8A/C5dxKYD/T1pl/glfYzgW+Bc0rblyX6yufTT12j3dVXR3e7q1erXnaZ+5Y2aaL6+OOu1G+Kt22ba+s45JDy/7KKxG+/qQ4a5D6bDh0qXj0UDStXqt54o2qdOi6us892DaqxLG2vX6/6wguqAwe6Ov/8uv/zz1d9/nk3P14qnOjj+ajMiX7yZNVWrVzSa9XKvQ66nBzV1q3dz+Nt22Kzj3nzXE+e/J/f//1vYvxUTkTDhrnv34cfxmd/b73lGnv97Ir57beuB1dSkmtEvfxy114Ub7t2qb77ruqIEa6nErjj8oc/qD7wgOqiRbH93lqij4PJk/eXJPIfdeoEP9lfcYX7+RrrEl1envtZ3KGDO7Y9e6p+9VVs91nZvPiiOzZ33RXf/W7dqnrTTfHtipmXpzprlurpp7v3XL++6xL5yy+x33ck8vJcAWXcONW0tP054cgjXU+f2bOj33XTEn0ctGpVOMnnP1q18juy2Pnvf917HDs2fvvcu9c1qB18sNv3xRer/vRT/PafqBYvdlUGJ58cnbro8vjqq9h3xdy7V/Xll1W7dXP7OeQQV1revDn6+4qm1atV//EP17WzRg0Xe0qK6pAhqv/+d3TaoCzRx4FI+EQv4ndksbF2rWvsSkvzpzFu61ZXcq1d2/1cv/XW8vcVr+x27lTt0sV9HllZ/sayZ4/ruhjtrpjbt6s+8YSrJsy/5uKZZ1R//73i2463rVtVX3nFVbM1aeLeT/Xqqn36uAsHy8sSfRxUpRJ9Xp7qWWe5nh2LFvkbS1aW6pVXuhNqSorq3/5WOf/5K2LECPdde/ttvyPZL7Qr5kknlX9ogfXr3Qk9PyH27Ol6dlWmfvwlyc11nRn+9Cd38urTp/zbskQfB1Wpjj6/j/bjj/sdyX6Zme5iFlBt08b9HK4KDbbTp7v3PHq035EcKC9P9dln93fFHD8+8l9/y5e7xt1atdz7GzBA9bPPYhtvIqhIhwZL9HFSFXrdLF3qTmCnn56Ypar33nPVGKB6/PHBTg4//aTaoIHrJ55ofdlD/faba0sB1Y4dVb/4ovhl5851V/RWq+ZODlddlfgXZiUKS/QmKvbudUklJcX/uuCS5Oa6i3oOO8x9wwcOdCeoINm9W/W441w9+MqVfkcTmdCumDfcsL8rZl6e6syZ+7vQNmjgLpL79Vd/461sLNGbqBg/3n1jpk3zO5LIbN/uYq5b1zV2/fnPiV3yLYtbb3WfxYwZfkdSNkW7Yt5///5fYM2bu6GCgzwscixZoo+jqVNdF7NEahiLhm++cRekDBnidyRlt3ata7AFN7DVsmV+R1Qxb77p3ssNN/gdSfmFdsXs0MFdXRqUk7BfSkr04uYnjvT0dM3IyPA7jDLLzXX34Pzb36B2bdi9Gx5/3N1Or7LbudPdcWjnTneP0UaN/I6ofGbMgKuvdp/VP/8JQ4b4HVHZZWW52/21bAlffgm1avkdUfnt2QPLl0P79lDNbmpaYSIyT1XTw82zwxsF69fD6ae7JH/DDfDrr3DWWTByJNxyC+zb53eEFfPnP8PSpfDii5U3yQNccIG7X21qKgwdCsOGwfbtfkcVudxcd9/X3bvdLQErc5IHqFEDOnSwJB8XxRX1/XpUtqqbr79WbdHCdQN74YX903NzVW+5xf00Peec2I0DE2vvvuvew6hRfkcSPZLXMjwAABFmSURBVHv3qv71r66euF07d6l6ZTB2rPssgtiby1QcVkcfG88847qAtWpVfLJ48knXVaxbt8rXi2DDBtVDD3Vd4vwaejWWZs92DYDJye7OWInc737WLHdiuvJKvyMxiaqkRG8/msph92649lpX33vSSZCRAWlp4Ze94QZ4801Ytgx69HBVB5WBKlx3HWzYAJMnV/5qgnB694b586FfPxg1Cs4+G7Kz/Y7qQL/95qqa2rd37T7GlJUl+jLKynLJfdIk+MtfYOZMaNq05HXOPBM++8wlz1694J134hNrRUyeDK+8AvfcA127+h1N7DRtCq+/7hLoBx+4+vuPPvI7qv3y8uDSSyEnB6ZPh7p1/Y7IVEaW6Mvgk0/g2GNh0SKXBB94AJKSIls3NRW+/hratYNzzoGnnoptrBXx88+uIblXLxg92u9oYk8EbrwRvvkGGjSAPn3gjjtg716/I3PfsQ8+gCeegE6d/I7GVFrF1en49UjEOvq8PFeHm5TkbmC9cGH5t7Vtm2ucBdWbb/ZvSNni7NvnhrqtX191xQq/o4m/7dv397k//nh/rzqdM8e17wwalNjtByYxYI2x5bd9u+oll2jBwErRGDc6kXvkPPSQi+v55/2OxF9Tp7pL8Rs2dAOHxduGDa431xFH2JWiJjIVTvRAP2AJsBwYE2Z+S2A28B2wADgzZN7t3npLgDNK21ciJfrly93l2SKq994b/UG8Eq1HTmam60V0/vlWglR1g4b16OH+S66+WnXHjvjsNy/P3e80OVk1IyM++zSVX4USPZCEuyl4W6AG7kbfHYosMwkY4T3vAKwKeZ4J1ATaeNtJKml/iZLo33lHtVEjN4DXzJmx28/bb6vWq+dKb/Pnx24/pfn9d3dJ+sEHx/eGxoluzx43wBaoHnOMOxnG2iOPuP099ljs92WCo6REH0ljbHdguaquUNU9wDRgQNGqfqCB97whsMZ7PgCYpqq7VXWlV7LvHsE+fZOX53qanHUWtGrluk6ecUbs9pcoPXL++lc3vMGzz0KzZv7EkIiSk12D6Pvvw+bN0L27a0jXGI0cMneu6801YIBrIDYmGiJJ9M2BX0JeZ3nTQt0NDBWRLOAdIP8rGsm6CSMnB84/H+66Cy65BL74Atq2jf1+/e6R88kn8PDD7tqAs86K774ri9NPd9dAnHKK65E0cCBs2hTdfeTkwMUXw6GHwnPPud5AxkRDJIk+3NetaHlmMPCCqrYAzgT+JSLVIlwXERkuIhkikpHt0xUrCxfCcce5EvXjj8O//gV16sRv/82bw5w5+8fIGTUqPmPk5OTAZZfBEUe4sXpM8Q46CN5+2x2nt992J+g5c6KzbVV3Ad7q1TB1KjRuHJ3tGgORJfos4PCQ1y3YXzWT7ypgOoCqfgnUAppGuC6qOklV01U1vZkP9Qb/+Y+7anXrVvjwQ/eT2Y/SVL168OqrcPPN8OijrtQY60G3brrJDcI2ebJdjBOJatXg1lv3jxx5yilw991uwLGKmDjRXZsxYQL84Q9RCdWY/YqrvNf9Da3VgRW4xtT8xtiORZZ5F7jce34MLpkL0JHCjbErSKDG2L173c0o8vtMJ9Jdk554IvY9cv7zH/fe77orNtsPuq1bVS+91B3DE09UXb26fNvJzFStWdPd8zYRb89oKgei0L3yTGAprtfMWG/aeOBc3d+75nMvqc8H+oasO9ZbbwnQv7R9xSvRZ2ernnaaOwLXXed6nSSat96KXY+cX39VbdzY3Y5uz57obruqeekl9zmlpKi++mrZ1t22TfXoo1UPOUR13brYxGeqhgon+ng+4pHoMzJUW7Z0pahnn4357ipk/nw3wmK9etG7a1Venis91q6t+uOP0dlmVbd0qeqxx7r/qOuvV925M7L1LrvMXafx0Uexjc8EX0mJvsqNdfPCC9Czp2v8+uwzuPJKvyMqWdEeOf/4R8W3+fTT8N57rqfN0UdXfHvGfT5ffOHq7//xD9fms2hRyeu8+CK89JLr2nrKKfGJ01RRxZ0B/HrEqkS/e7fqiBGuxHXqqZXvoqDQMXJuuaX8Y+T8+KMryZ9xhl39GivvvKParJk7zpMmhT/Oixer1qnjxhVKtPGOTOVEVS/Rr1njxh5/+mn4059cabayXRQUjR45e/e6cc1r17Z+2rHUv7/rc9+zJwwf7vrGb9myf/6uXXDRRa777pQpkY+Aakx5BT7Rf/qpuynIggVuPO8HH4Tq1f2OqnySklySf+IJeOstOPlkdxKL1L33uit9J06Eww6LXZzGXfT03nvuqtpXX3Vj+n/5pZt3663uKuSXXnLXTxgTc8UV9f16RKvqJi9P9fHHVatXd/cF/eGHqGw2YZS1R86XX7phli+7LPaxmcK+/FK1dWt3/PNHQv3Tn/yOygQNVa3qZudOGDbMXQzUv7+7oUTHjn5HFV1nneV+rWgEY+Ts2OHuUtS8ud2Kzg/HH+9uWXjhhfDyy66hdsIEv6MyVUngEv3Kla5udPJkGD8eXnsNGjXyO6rY6No1sh45t90GP/3kqgoaNoxvjMZp2NANbTBrlqt2S072OyJTlVTS2urw3n8fBg1ypdy33nIjQwZd/hg5gwe7G5EvW+a6TeY38L39tquTHz3a1ekb/4i42xQaE2+BKdH/+KOrpmnRwg31WhWSfL569dwvl5tuKtwjJzsbrroKOnd2DbHGmKopMCX69u3diJMDBlTNwbmSkuCxx1w1zs03u9L7wQe7MdTffx9q1vQ7QmOMXwKT6MGNIV/VjRwJbdq4vtvffgsPPQRduvgdlTHGT4FK9MY56yx3Of6sWXDLLX5HY4zxmyX6gOrSxUryxhgnMI2xxhhjwrNEb4wxAWeJ3hhjAs4SvTHGBJwlemOMCbiIEr2I9BORJSKyXETGhJn/dxGZ7z2WisiWkHn7Qua9Ec3gjTHGlK7U7pUikgQ8BZwOZAFzReQNVS24UZqqjgpZ/kagW8gmdqlq1+iFbIwxpiwiKdF3B5ar6gpV3QNMAwaUsPxgYGo0gjPGGFNxkST65sAvIa+zvGkHEJFWQBvgo5DJtUQkQ0S+EpHzyh2pMcaYconkythwdxbVYpYdBLyiqvtCprVU1TUi0hb4SES+V9WfCu1AZDgwHKBly5YRhGSMMSZSkZTos4DDQ163AIq7U+kgilTbqOoa7+8K4GMK19/nLzNJVdNVNb1ZZbtrtzHGJLhIEv1coJ2ItBGRGrhkfkDvGRE5GkgBvgyZliIiNb3nTYGewKKi6xpjjImdUqtuVDVXREYC7wFJwHOqulBExuNuRpuf9AcD07yb1OY7BpgoInm4k8oDob11jDHGxJ4Uzsv+S09P14yMDL/DMMaYSkVE5qlqerh5dmWsMcYEnCV6Y4wJOEv0xhgTcJbojTEm4CzRG2NMwFmiN8aYgLNEb4wxAWeJ3hhjAs4SvTHGBJwlemOMCThL9MYYE3CW6I0xJuAs0RtjTMBZojfGmICzRG+MMQFnid4YYwLOEr0xxgScJXpjjAk4S/TGGBNwESV6EeknIktEZLmIjAkz/+8iMt97LBWRLSHzhonIMu8xLJrBG2OMKV310hYQkSTgKeB0IAuYKyJvqOqi/GVUdVTI8jcC3bznjYFxQDqgwDxv3c1RfRfGGGOKFUmJvjuwXFVXqOoeYBowoITlBwNTvednALNUdZOX3GcB/SoSsDHGmLKJJNE3B34JeZ3lTTuAiLQC2gAflWVdERkuIhkikpGdnR1J3MYYYyIUSaKXMNO0mGUHAa+o6r6yrKuqk1Q1XVXTmzVrFkFIxhhjIhVJos8CDg953QJYU8yyg9hfbVPWdY0xxsRAJIl+LtBORNqISA1cMn+j6EIicjSQAnwZMvk9oK+IpIhICtDXm2aMMSZOSu11o6q5IjISl6CTgOdUdaGIjAcyVDU/6Q8Gpqmqhqy7SUTuwZ0sAMar6qbovgVjjDElkZC8nBDS09M1IyPD7zCMMaZSEZF5qpoebp5dGWuMMQFnid4YYwLOEr0xxgScJXpjjAk4S/TGGBNwluiNMSbgLNEbY0zAWaI3xpiAs0RvjDEBZ4neGGMCzhK9McYEnCV6Y4wJOEv0xhgTcJbojTEm4CzRG2NMwFmiN8aYgLNEb4wxAWeJ3hhjAi6iRC8i/URkiYgsF5ExxSxzkYgsEpGFIvJyyPR9IjLfexxwU3FjjDGxVerNwUUkCXgKOB3IAuaKyBuquihkmXbA7UBPVd0sIgeFbGKXqnaNctzGGGMiFEmJvjuwXFVXqOoeYBowoMgy1wBPqepmAFVdH90wjTHGlFckib458EvI6yxvWqijgKNE5HMR+UpE+oXMqyUiGd7088LtQESGe8tkZGdnl+kNGGOMKVmpVTeAhJmmYbbTDugNtAA+FZFOqroFaKmqa0SkLfCRiHyvqj8V2pjqJGASQHp6etFtG2OMqYBISvRZwOEhr1sAa8Is87qq7lXVlcASXOJHVdd4f1cAHwPdKhizMcaYMogk0c8F2olIGxGpAQwCivaeeQ04BUBEmuKqclaISIqI1AyZ3hNYhDHGmLgptepGVXNFZCTwHpAEPKeqC0VkPJChqm948/qKyCJgH/AnVd0oIn8AJopIHu6k8kBobx1jjDGxJ6qJVSWenp6uGRkZfodhjDGViojMU9X0cPPsylhjjAk4S/TGGBNwluiNMSbgLNEbY0zAWaI3xpiAC0yinzIFWreGatXc3ylT/I7IGGMSQyRDICS8KVNg+HDYudO9/vln9xpgyBD/4jLGmEQQiBL92LH7k3y+nTvddGOMqeoCkehXry7bdGOMqUoCkehbtizbdGOMqUoCkegnTIA6dQpPq1PHTTfGmKouEIl+yBCYNAlatQIR93fSJGuINcYYCEivG3BJ3RK7McYcKBAlemOMMcWzRG+MMQFnid4YYwLOEr0xxgScJXpjjAm4hLuVoIhkAz/7HUcFNQU2+B1EArHjUZgdj/3sWBRWkePRSlWbhZuRcIk+CEQko7h7N1ZFdjwKs+Oxnx2LwmJ1PKzqxhhjAs4SvTHGBJwl+tiY5HcACcaOR2F2PPazY1FYTI6H1dEbY0zAWYneGGMCzhK9McYEnCX6KBKRw0VktogsFpGFInKz3zH5TUSSROQ7EXnL71j8JiKNROQVEfnR+46c4HdMfhKRUd7/yQ8iMlVEavkdUzyJyHMisl5EfgiZ1lhEZonIMu9vSjT2ZYk+unKB21T1GOB44AYR6eBzTH67GVjsdxAJ4jFgpqq2B1KpwsdFRJoDNwHpqtoJSAIG+RtV3L0A9CsybQzwoaq2Az70XleYJfooUtW1qvqt93wb7h+5ub9R+UdEWgBnAc/4HYvfRKQBcBLwLICq7lHVLf5G5bvqQG0RqQ7UAdb4HE9cqeocYFORyQOAF73nLwLnRWNfluhjRERaA92Ar/2NxFePAn8G8vwOJAG0BbKB572qrGdEpK7fQflFVX8FHgZWA2uBHFV939+oEsLBqroWXMEROCgaG7VEHwMiUg+YAdyiqlv9jscPInI2sF5V5/kdS4KoDqQBT6tqN2AHUfpZXhl5dc8DgDbAYUBdERnqb1TBZYk+ykQkGZfkp6jqf/2Ox0c9gXNFZBUwDThVRCb7G5KvsoAsVc3/hfcKLvFXVX2Alaqarap7gf8Cf/A5pkSwTkQOBfD+ro/GRi3RR5GICK4OdrGqPuJ3PH5S1dtVtYWqtsY1sn2kqlW2xKaqvwG/iMjR3qTTgEU+huS31cDxIlLH+785jSrcOB3iDWCY93wY8Ho0NhqYm4MniJ7ApcD3IjLfm3aHqr7jY0wmcdwITBGRGsAK4Aqf4/GNqn4tIq8A3+J6q31HFRsOQUSmAr2BpiKSBYwDHgCmi8hVuJPhH6OyLxsCwRhjgs2qbowxJuAs0RtjTMBZojfGmICzRG+MMQFnid4YYwLOEr0xxgScJXpjjAm4/wfpiYiUJLkrNAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxU5dn/8c/FIoggKGBVQIJLVTYhRsSCAq64gEtdQCiuRa1aFTfqXqs/fZQqotSKWvURlCJWpVbFR6EiVoWwKiCiCBJBCBQQBIUk1++PexIChGSSzMxJJt/365UXM2fO3OeaM8w199zbMXdHRESqv1pRByAiIomhhC4ikiaU0EVE0oQSuohImlBCFxFJE0roIiJpQgldSmRmtc1so5kdkMh9o2RmB5tZwsfpmtmJZrak2P2FZnZsPPtW4FjPmNltFX1+KeXeZ2bPJ7pcSa06UQcgiWFmG4vdbQD8DOTH7l/h7mPKU5675wMNE71vTeDuhyaiHDO7HBjo7j2LlX15IsqW9KSEnibcvSihxmqAl7v7e7va38zquHteKmITkdRQk0sNEftJ/Xcze9nMNgADzewYM/vEzNaZ2QozG2FmdWP71zEzN7OM2P3RscffNrMNZvaxmbUp776xx081sy/NbL2ZPW5mH5nZxbuIO54YrzCzr8xsrZmNKPbc2mb2qJmtMbOvgd6lnJ87zGzsDttGmtkjsduXm9mC2Ov5OlZ73lVZOWbWM3a7gZm9GIttHnBkCcddHCt3npn1jW3vADwBHBtrzlpd7NzeU+z5V8Ze+xoze93M9ovn3JTFzM6KxbPOzCaZ2aHFHrvNzJab2Q9m9kWx19rVzGbGtq80s4fjPZ4kiLvrL83+gCXAiTtsuw/YAvQhfJHvDhwFHE34pXYg8CVwTWz/OoADGbH7o4HVQBZQF/g7MLoC++4DbADOjD02BNgKXLyL1xJPjG8AjYEM4L+Frx24BpgHtASaAlPCf/kSj3MgsBHYo1jZq4Cs2P0+sX0MOB7YDHSMPXYisKRYWTlAz9jtYcC/gb2A1sD8HfY9H9gv9p5cGIvhF7HHLgf+vUOco4F7YrdPjsXYCagP/AWYFM+5KeH13wc8H7t9eCyO42Pv0W2x814XaAcsBfaN7dsGODB2ezrQP3a7EXB01J+FmvanGnrNMtXd/+nuBe6+2d2nu/un7p7n7ouBUUCPUp4/3t2z3X0rMIaQSMq77xnAbHd/I/bYo4TkX6I4Y3zA3de7+xJC8iw81vnAo+6e4+5rgAdLOc5i4HPCFw3AScA6d8+OPf5Pd1/swSTgfaDEjs8dnA/c5+5r3X0podZd/Ljj3H1F7D15ifBlnBVHuQADgGfcfba7/wQMBXqYWcti++zq3JSmHzDB3SfF3qMHgT0JX6x5hC+PdrFmu29i5w7CF/MhZtbU3Te4+6dxvg5JECX0mmVZ8TtmdpiZ/cvMvjezH4B7gWalPP/7Yrc3UXpH6K723b94HO7uhBptieKMMa5jEWqWpXkJ6B+7fSHhi6gwjjPM7FMz+6+ZrSPUjks7V4X2Ky0GM7vYzObEmjbWAYfFWS6E11dUnrv/AKwFWhTbpzzv2a7KLSC8Ry3cfSFwI+F9WBVrwts3tuslQFtgoZlNM7PT4nwdkiBK6DXLjkP2niLUSg929z2BuwhNCsm0gtAEAoCZGdsnoB1VJsYVQKti98saVvl34MRYDfdMQoLHzHYHxgMPEJpDmgDvxhnH97uKwcwOBJ4ErgKaxsr9oli5ZQ2xXE5oxiksrxGhaee7OOIqT7m1CO/ZdwDuPtrduxGaW2oTzgvuvtDd+xGa1f4MvGpm9SsZi5SDEnrN1ghYD/xoZocDV6TgmG8CmWbWx8zqANcBzZMU4zjgejNrYWZNgVtL29ndVwJTgeeAhe6+KPZQPWA3IBfIN7MzgBPKEcNtZtbEwjj9a4o91pCQtHMJ322XE2rohVYCLQs7gUvwMnCZmXU0s3qExPqhu+/yF085Yu5rZj1jx76Z0O/xqZkdbma9YsfbHPvLJ7yA35hZs1iNfn3stRVUMhYpByX0mu1G4CLCh/UpQg01qWJJ8wLgEWANcBAwizBuPtExPklo6/6M0GE3Po7nvETo5HypWMzrgBuA1wgdi+cSvpjicTfhl8IS4G3gf4uVOxcYAUyL7XMYULzd+f+ARcBKMyvedFL4/HcITR+vxZ5/AKFdvVLcfR7hnD9J+LLpDfSNtafXAx4i9Ht8T/hFcEfsqacBCyyMohoGXODuWyobj8TPQhOmSDTMrDbhJ/657v5h1PGIVGeqoUvKmVlvM2sc+9l+J2HkxLSIwxKp9pTQJQrdgcWEn+29gbPcfVdNLiISJzW5iIikCdXQRUTSRGSLczVr1swzMjKiOryISLU0Y8aM1e5e4lDfyBJ6RkYG2dnZUR1eRKRaMrNdznhWk4uISJpQQhcRSRNK6CIiaUJXLBKpIbZu3UpOTg4//fRT1KFIHOrXr0/Lli2pW3dXS/nsTAldpIbIycmhUaNGZGRkEBa5lKrK3VmzZg05OTm0adOm7CfEVKsmlzFjICMDatUK/44p12WPRWq2n376iaZNmyqZVwNmRtOmTcv9a6ra1NDHjIHBg2HTpnB/6dJwH2BApdeXE6kZlMyrj4q8V9Wmhn777duSeaFNm8J2ERGpRgn922/Lt11EqpY1a9bQqVMnOnXqxL777kuLFi2K7m/ZEt+y6ZdccgkLFy4sdZ+RI0cyJkHtsd27d2f27NkJKSsVqk2TywEHhGaWkraLSOKNGRN+AX/7bfic3X9/5Zo3mzZtWpQc77nnHho2bMhNN9203T5FV6+vVXJd87nnnivzOFdffXXFg6zmqk0N/f77oUGD7bc1aBC2i0hiFfZZLV0K7tv6rJIxEOGrr76iffv2XHnllWRmZrJixQoGDx5MVlYW7dq149577y3at7DGnJeXR5MmTRg6dChHHHEExxxzDKtWrQLgjjvuYPjw4UX7Dx06lC5dunDooYfyn//8B4Aff/yRX//61xxxxBH079+frKysMmvio0ePpkOHDrRv357bbrsNgLy8PH7zm98UbR8xYgQAjz76KG3btuWII45g4MCBCT9nu1JtEvqAATBqFLRuDWbh31Gj1CEqkgyp7rOaP38+l112GbNmzaJFixY8+OCDZGdnM2fOHP7v//6P+fPn7/Sc9evX06NHD+bMmcMxxxzD3/72txLLdnemTZvGww8/XPTl8Pjjj7PvvvsyZ84chg4dyqxZs0qNLycnhzvuuIPJkycza9YsPvroI958801mzJjB6tWr+eyzz/j8888ZNGgQAA899BCzZ89mzpw5PPHEE5U8O/GrNgkdQvJesgQKCsK/SuYiyZHqPquDDjqIo446quj+yy+/TGZmJpmZmSxYsKDEhL777rtz6qmnAnDkkUeyZMmSEss+55xzdtpn6tSp9OvXD4AjjjiCdu3alRrfp59+yvHHH0+zZs2oW7cuF154IVOmTOHggw9m4cKFXHfddUycOJHGjRsD0K5dOwYOHMiYMWPKNTGosqpVQheR1NhV31Sy+qz22GOPotuLFi3iscceY9KkScydO5fevXuXOB57t912K7pdu3Zt8vLySiy7Xr16O+1T3gv77Gr/pk2bMnfuXLp3786IESO44oorAJg4cSJXXnkl06ZNIysri/z8/HIdr6KU0EVkJ1H2Wf3www80atSIPffckxUrVjBx4sSEH6N79+6MGzcOgM8++6zEXwDFde3alcmTJ7NmzRry8vIYO3YsPXr0IDc3F3fnvPPO449//CMzZ84kPz+fnJwcjj/+eB5++GFyc3PZtGP7VZJUm1EuIpI6hc2ZiRzlEq/MzEzatm1L+/btOfDAA+nWrVvCj3HttdcyaNAgOnbsSGZmJu3bty9qLilJy5Ytuffee+nZsyfuTp8+fTj99NOZOXMml112Ge6OmfE///M/5OXlceGFF7JhwwYKCgq49dZbadSoUcJfQ0niuqaomfUGHgNqA8+4+4Ml7HM+cA/gwBx3v7C0MrOyslwXuBBJnQULFnD44YdHHUaVkJeXR15eHvXr12fRokWcfPLJLFq0iDp1qlYdt6T3zMxmuHtWSfuXGb2Z1QZGAicBOcB0M5vg7vOL7XMI8Aegm7uvNbN9KvEaRESSauPGjZxwwgnk5eXh7jz11FNVLplXRDyvoAvwlbsvBjCzscCZQPFGp98CI919LYC7r0p0oCIiidKkSRNmzJgRdRgJF0+naAtgWbH7ObFtxf0S+KWZfWRmn8SaaHZiZoPNLNvMsnNzcysWsYiIlCiehF7Skl87NrzXAQ4BegL9gWfMrMlOT3If5e5Z7p7VvHmJF60WEZEKiieh5wCtit1vCSwvYZ833H2ru38DLCQkeBERSZF4Evp04BAza2NmuwH9gAk77PM60AvAzJoRmmAWJzJQEREpXZkJ3d3zgGuAicACYJy7zzOze82sb2y3icAaM5sPTAZudvc1yQpaRKqfnj177jRJaPjw4fzud78r9XkNGzYEYPny5Zx77rm7LLusYdDDhw/fboLPaaedxrp16+IJvVT33HMPw4YNq3Q5iRDXTFF3f8vdf+nuB7n7/bFtd7n7hNhtd/ch7t7W3Tu4+9hkBi0i1U///v0ZO3b71DB27Fj69+8f1/P3339/xo8fX+Hj75jQ33rrLZo02amrr1rT1H8RSYlzzz2XN998k59//hmAJUuWsHz5crp37140LjwzM5MOHTrwxhtv7PT8JUuW0L59ewA2b95Mv3796NixIxdccAGbN28u2u+qq64qWnr37rvvBmDEiBEsX76cXr160atXLwAyMjJYvXo1AI888gjt27enffv2RUvvLlmyhMMPP5zf/va3tGvXjpNPPnm745Rk9uzZdO3alY4dO3L22Wezdu3aouO3bduWjh07Fi0K9sEHHxRd4KNz585s2LChwue2UPUfSS8i5Xb99ZDoC/F06gSxXFiipk2b0qVLF9555x3OPPNMxo4dywUXXICZUb9+fV577TX23HNPVq9eTdeuXenbt+8ur6v55JNP0qBBA+bOncvcuXPJzMwseuz+++9n7733Jj8/nxNOOIG5c+fy+9//nkceeYTJkyfTrFmz7cqaMWMGzz33HJ9++inuztFHH02PHj3Ya6+9WLRoES+//DJPP/00559/Pq+++mqp65sPGjSIxx9/nB49enDXXXfxxz/+keHDh/Pggw/yzTffUK9evaJmnmHDhjFy5Ei6devGxo0bqV+/fjnOdslUQxeRlCne7FK8ucXdue222+jYsSMnnngi3333HStXrtxlOVOmTClKrB07dqRjx45Fj40bN47MzEw6d+7MvHnzylx4a+rUqZx99tnsscceNGzYkHPOOYcPP/wQgDZt2tCpUyeg9CV6IazPvm7dOnr06AHARRddxJQpU4piHDBgAKNHjy6akdqtWzeGDBnCiBEjWLduXUJmqqqGLlIDlVaTTqazzjqLIUOGMHPmTDZv3lxUsx4zZgy5ubnMmDGDunXrkpGRUeKSucWVVHv/5ptvGDZsGNOnT2evvfbi4osvLrOc0tazKlx6F8Lyu2U1uezKv/71L6ZMmcKECRP405/+xLx58xg6dCinn346b731Fl27duW9997jsMMOq1D5hVRDF5GUadiwIT179uTSSy/drjN0/fr17LPPPtStW5fJkyeztKQLCBdz3HHHFV0I+vPPP2fu3LlAWHp3jz32oHHjxqxcuZK333676DmNGjUqsZ36uOOO4/XXX2fTpk38+OOPvPbaaxx77LHlfm2NGzdmr732Kqrdv/jii/To0YOCggKWLVtGr169eOihh1i3bh0bN27k66+/pkOHDtx6661kZWXxxRdflPuYO1INXURSqn///pxzzjnbjXgZMGAAffr0ISsri06dOpVZU73qqqu45JJL6NixI506daJLly5AuPpQ586dadeu3U5L7w4ePJhTTz2V/fbbj8mTJxdtz8zM5OKLLy4q4/LLL6dz586lNq/sygsvvMCVV17Jpk2bOPDAA3nuuefIz89n4MCBrF+/HnfnhhtuoEmTJtx5551MnjyZ2rVr07Zt26KrL1VGXMvnJoOWzxVJLS2fW/2Ud/lcNbmIiKQJJXQRkTShhC5Sg0TVxCrlV5H3SgldpIaoX78+a9asUVKvBtydNWvWlHuykUa5iNQQLVu2JCcnB11cpnqoX78+LVu2LNdzlNBFaoi6devSpk2bqMOQJFKTi4hImlBCFxFJoXXrIC8vOWUroYtIUi1YAJpDCGvXwl13QevW8NJLyTmGErqIJM0//wlHHglHHQW9esF770FNG2Tz3//CnXdCRgb86U9w4olQbLXfhFJCF5Gk+Otf4ayzoF07eOgh+PJLOOkk6NoV3ngDCgqijjC51qyBO+4Iify++8JrnzMHXn0VYtfpSDgldBFJKHe47Ta46io49VT497/h5pth8WJ46ilYvTok+iOOgJdfhvz8qCNOrDVr4PbbQyK//37o3RvmzoXx46HYsu1JoYQukmAvvwwHHwxvvhl1JKm3ZQsMGgQPPAC//S28/jrssUd4rF49GDwYFi6E0aNDDf3CC+Gww+CZZ8Jzq7PVq8MXWUZGeP2nnQaffQbjxkGHDikKwt0j+TvyyCNdJN1MmuRet657vXru4H7nne55eVFHlRrr1rmfcEJ43ffd515QUPr++fnu//iH+5FHhue0bOk+fLj7jz+mJt5EWbXK/dZb3ffYw93M/YIL3D//PHnHA7J9F3lVCV0kQebNc2/c2L1tW/cVK9wvuSR8wk45xX316qijS65ly9w7dHCvU8f9hRfK99yCAveJE92POy6cr+bN3f/f/wtfEFXZqlXut9yyLZH36xf+DySbErpIkq1Y4d66tfu++7ovWRK2FRS4jxrlvttu4bHs7CgjTJ65c91btHBv1Mj93XcrV9aHH7r37h0yU+PG7rff7p6bm5g4E2XlSvebb3Zv0CAk8v793efPT93xldBFkmjDhtBs0KBByUl72jT3Vq1CM8wzz6Q+vmR6/333Pfd0339/99mzE1fujBnuv/51SJgNGrjfcIN7Tk7iyq+IlSvdb7opxFOrlvuAAe4LFqQ+jkondKA3sBD4ChhawuMXA7nA7Njf5WWVqYQu6WDrVvczzggf8H/+c9f75ea6n3RS+MRdfrn75s2pizFZRo8O/QXt2rl/+21yjjF/vvugQe61a4dfOoMHu3/9dXKOtSsrVrgPGeK+++7hfR440P2LL1IbQ3GVSuhAbeBr4EBgN2AO0NZ3TuhPlFWWK6FLGikocP/d78Kn6C9/KXv/vLzQhAChRl/YNFPdFBSENm5w79nTfe3a5B9z8WL3K68MSb127ZBUk9nx6B4S+Q03bEvkv/mN+8KFyT1mPCqb0I8BJha7/wfgDzvso4QuNc6wYeETdPPN5XveG2+EZoq99w6dgdXJ1q0hsUJoO/7pp9Qe/7vv3G+8MXREgvvZZye+b2L5cvfrr3evXz8k8kGD3L/8MrHHqIzKJvRzgWeK3f/Njsk7ltBXAHOB8UCrXZQ1GMgGsg844IDUnQGRBHvllfDpOe+8MPyuvL78MowKMQtD/CpSRqpt3Ojep0943bfeGm3Mq1eHIaFNmnjRSKIPPqhcmcuXu193XUjktWu7X3yx+6JFiYk3kSqb0M8rIaE/vsM+TYF6sdtXApPKKlc1dKmuPvoodHD+6leVawvfuDF0rEFIlKlouqiolSvdu3QJNdaRI6OOZpv1690ffNB9n33Ceeze3f2tt8oeA19cTo77tdeG97R27TDc9KuvkhdzZSW9yWWH/WsD68sqVwldqqMvv3Rv2tT9kEMSM5yuoMD9iSfC+O2DDnKfM6fyZSbal1+G2Hbf3f3116OOpmQ//ug+YkQYTQTumZnu48eX/isiJ8f9mmu2JfJLL63aibxQZRN6HWAx0KZYp2i7HfbZr9jts4FPyiq3ogl91Sr3p56q0FNFKiU31/3gg92bNUv8T/GPPgpD/3bf3f3FFxNbdmV8/HF4vc2ahdtV3c8/uz/7bPjCBffDDgsTnbZs2bbPsmXuV18dOljr1HG/7LLUj5ypjEQMWzwN+DI22uX22LZ7gb6x2w8A82LJfjJwWFllVjSh33VXiHro0PL9rBKpjE2b3I85JrSv/uc/yTnGihXuPXqE/99XXx2SU5Refz283oMOqlqdgvHIy3MfO9a9Y8dwPjMy3B9/PIxKKkzkl18eRs9UN2k1sSgvz/2KK0LkV11VPTqTpHrLzw+dn2bhZ3wybd0aJq+Ae9euoTYZhSeeCO3lXbqE9vPqqqDAfcIE96OPDue0Tp0wlv2bb6KOrOLSKqG7hzfplltC9AMGbP9zSiTRChPssGGpO+Yrr7g3bBg6+yZNSt1x8/O3fbb69Akdt+mgoMB9+nT3pUujjqTy0i6hFyqc3NC3b3rMvJOqZ+TIbU0gqW7imz8/tAHXquX+0EPJP/5PP4Wx5YW/frduTe7xpGLSNqG7b/vAHX+8+w8/JKTIauObb8JPyf79a84Srak0YUJIpn36RHd+f/jB/dxzw//xc84Jw/SSYe3aMOsT3B94QP1TVVlaJ3T3MCqgdu2Q3NasSVixVdqkSWH43O67b6tR6UOYONOnh0WYsrKib3YoKHD/85/D//FDD038Eq1Ll4b1WOrWDeuzSNVWWkJPiysWDRwYrtM3axb06AErVkQdUfK4w4gR4fqE++wTrlF4883w5JPw4INRR5celiyBM86A5s3DRY4Lr7gTFTMYMgTefz9cOb5Ll3AVnESYMweOOQaWLYN33oEBAxJTrkRkV5k+2X/JmFj03nthjYeDDqrevdi7snlzmI5c2G9Q+PM7P39b22d5Ly4g21u71v3ww8OU8lSucR2vnJwwQxXCCoCVGRDw7rthDfOWLcOa5lI9kO5NLsV9/HH4MLZoUTU/kBX13Xfbhl7dddfOwzV/+in0I9SpU/0WfKoqfv7ZvVev0PSQypEl5fXzz2GqOoSr/KxYUf4yXngh/F/p0CG6oZFSMTUqobuH6dO/+EWY3TZjRtIOkzIff+y+337h18err+56v3XrwkSKhg3dZ85MXXzpoKAgLI8KVWumZmnGjAnt/Pvt5z51anzPKSgIi4FBuP5nVb/Mm+ysxiV09zCz7YADwjKlU6Yk9VBJ9eyzYWbbgQe6f/ZZ2fvn5IT1LPbdNz2bnZLlzjvDp+FPf4o6kvKZOzcsR1Cnjvtjj5XeMb51q/tvfxte58CB0c9ElYqpkQndPVxF5dBDw0iQt99O+uESasuWsHAQhCvdlGf0zrx5odnp0EPT/+LEifDss+E8X3ZZ9RwptHZt6FMpXKO8pFE5Gza4n3Za2Oe226rn65SgxiZ09zBtuVOn0C46blxKDllpq1ZtGxN8440Vm+DxwQehZv+rX4V1SKRk774barcnn1y9Zxzn54eJdrVqubdvv/2Vdb7/PlwhqVYt97/+NboYJTFqdEJ3DzWYbt3Cf+hnn03ZYStk1qxwhfh69SrfljtuXFh/5OyzNfGoJHPnhlEeHTokb8JOqr37bpifsOee7q+9Fq592aZNaGsv7ZqnUn3U+ITuHtZLPuWU8IofeSSlh47b2LGheahlyzCxJRGGD/fIpq5XZTk54Tzvv3/6jfJYujRMiILQQd68ufu0aVFHJYlSWkJPi4lF8WjQAN54A3796zBJ4557wiSdqiA/H/7wB+jXDzIzITsbsrISU/Z118GNN8LIkfDQQ4kps7rbsAFOPx3WrYN//Qtatow6osQ64ACYOhV+9zs47DD4+GM46qioo5KU2FWmT/ZfVFcs2ro1XGIKwvUDo15+d+1a91NPDfFccUVyRh7k57v36xeOUdOndm/Z4t67d5hG/847UUcjUn6UUkOvE/UXSqrVqQPPPAONG8Pw4bB+PTz9dNieagsWwJlnwjffwF//CldckZzj1KoFzz8PK1fCJZfAL34BJ56YnGNVZe5w9dVhivvTT8Mpp0QdkUhi1Zgml+Jq1YJHHgnNLs8/H5o6fv45tTH8859w9NHhC2Xy5OQl80L16sE//gGHHgrnnAOzZyf3eFXRgw+GRH7bbXD55VFHI5J4NTKhQ1jw6O674dFHw8JeffvCjz8m/7jucN99oWZ+yCGhvbx79+QfF6BJE3j77fDr5LTTYOnS1By3Knj55ZDIL7wwnH+RdFRjE3qh66+HZ5+F994LP8HXrUvesTZuhPPOgzvvDIll6lRo1Sp5xytJy5ahyWHTJjj1VPjvf1N7/ChMmQIXXwzHHQd/+1v4MhdJRzU+oQNcein8/e8wbRr06gWrViX+GIsXw69+Ba+9BsOGwYsvwu67J/448WjXLoz4+frr8Evhp5+iiSMVvvgCzjoL2rQJ575evagjEkkeJfSYc88N7doLF4aa3LJliSv7/ffDsLGcnNDkceON0dcSe/QIXypTp4b15PPzo40nGVauDE1LdeuG87733lFHJJJcSujFnHIKvPtuuEBG9+6waFHlynMPI2lOOQX22w+mT4eTT05MrIlw/vmhc/jVV+GGG6rOuPxE2LQp9It8/z28+WaooYukOyX0HXTvHkadbNoExx4Lc+dWrJyffgrttjfcEBLLxx/DQQclNNSEuOGG8Pf446EpKB3k54c+iunTQ2eoJtVITaGEXoLMTPjwwzA2vUcP+OST8j3/u+9Cs83//i/88Y8wfjw0apScWBNh2LBQW7/lFnjppaijqbwhQ0IfwWOPhT4CkZoiroRuZr3NbKGZfWVmQ0vZ71wzczNL0MT16Bx2WGhfbtYsTMJ57734nvef/8CRR4ZJQ6+9BnfdFca9V2W1aoUvnx49wq+KSZOijqjiHnssXHP1+uvh2mujjkYktcpMNWZWGxgJnAq0BfqbWdsS9msE/B74NNFBRiUjI9TUDzwwrP3xxhul7//009CzZ6iNf/JJGF1RXdSrB6+/Dr/8JZx9dsWbmqLgHprJzjgjJPKzz06f5iOR8oin7tgF+MrdF7v7FmAsUNIP2T8BDwFpNQhu333h3/+Gzp3Dwl4vvrjzPlu2hCnlgweHYY/TpoWhgdVN4cSjRo3CGPVvv406ooqWBBIAAA2hSURBVNJt3QpjxoSFzI4/Ppz3e+4J22rXjjo6kdSLJ6G3AIoP4suJbStiZp2BVu7+ZmkFmdlgM8s2s+zc3NxyBxuVvfcOTS49esCgQWHlwkKrVsFJJ8Ff/gI33wxvvQV77RVdrJXVqlVI6hs3hqS+dm3UEe1s/Xp4+OHwy2ngwNCBPWpUmPl6993Rje8XiVo8S1KVNGK6aICbmdUCHgUuLqsgdx8FjALIysqqVoPkGjYMS6326wfXXBOSSu/eoVklNzfUCi+8MOooE6NDh9D80rt36FR8912oXz/qqELCfuyxsLjahg3h19Bf/xq+eKp6P4VIKsTzMcgBik9QbwksL3a/EdAe+LeZLQG6AhPSoWN0R/XrwyuvQLducPvtofNz+fJwO12SeaFeveCFF0IfwqBBUFAQXSzTp4cv0oMOCh2effrAjBmh8/b005XMRQrFU0OfDhxiZm2A74B+QFH6cvf1QLPC+2b2b+Amd89ObKhVw7hxMHPmtvv5+fDAA2HiyoAB0cWVDP36hSGYN90E++8fFjJL1QzXgoIwc/fPfw5fKnvuGcbL//73qV//RqS6KLNu4+55wDXARGABMM7d55nZvWbWN9kBVjW33w6bN2+/bdOmsD0dDRkSrnr02GNhVmmybdoETz4Zho2edVZoZnnkkbAUw8MPK5mLlMY8ovneWVlZnp1d/SrxtWqVPEXeLNpmiWQqKAi19VdeCTMv+/VL/DFWroQnngjJfM2aMLvzxhvDyKIoLj4iUlWZ2Qx3L7FJWx+VcjrggJLXET/ggNTHkiqFE49WroSLLgpXPOrVKzFlz5sXauCjR4dhiH36hER+7LHRL2AmUt2oO6mc7r8/XHC6uAYNwvZ0Vr9+GPly8MGhKeSzzypelntYgfK006B9+7DcwKWXhqVu33gjLJugZC5Sfkro5TRgQBjz3Lp1SDqtW4f76dYhWpK99gpj1Bs2DEMFy7vE8JYtYWJW585hOYUZM+Dee0M5Tz4ZZqmKSMWpDV3Kbe7c0CTSqlVY76ZJk9L3X7s2fOmNGBGGeR5+eGhWGTCgaoxvF6lOSmtDVw1dyq1jx7Dw2JdfhuaXXV1g+5tvwgiZVq1g6NAwcuVf/4LPP4fLLlMyF0k0JXSpkOOPh+efhw8+2Hni0SefhGunHnxwWBLh7LNh1qxt7eaaCCSSHBrlIhV24YVh4tEtt0CLFqEZZtiwsIRw48ZhQtK114YLU4tI8imhS6XcdFPo1Hz00fCXkREuu3fppVX7oh4i6UgJXSrFLCTyFi3C8gfnnKOJQCJR0UdPKq12bbj11qijEBF1T4mIpAkldBGRNKGELiKSJpTQRUTShBK6iEiaUEIXEUkTSugiImlCCV1EJE0ooYuIpAkldBGRNKGELiKSJpTQRUTShBK6iEiaUEIXEUkTcSV0M+ttZgvN7CszG1rC41ea2WdmNtvMpppZ28SHKiIipSkzoZtZbWAkcCrQFuhfQsJ+yd07uHsn4CHgkYRHKiIipYqnht4F+MrdF7v7FmAscGbxHdz9h2J39wA8cSGKiEg84rliUQtgWbH7OcDRO+5kZlcDQ4DdgOMTEp2IiMQtnhq6lbBtpxq4u49094OAW4E7SizIbLCZZZtZdm5ubvkiFRGRUsWT0HOAVsXutwSWl7L/WOCskh5w91HunuXuWc2bN48/SinRmDGQkQG1aoV/x4yJOiIRiVI8CX06cIiZtTGz3YB+wITiO5jZIcXung4sSlyIUpIxY2DwYFi6FNzDv4MHK6mL1GRlJnR3zwOuASYCC4Bx7j7PzO41s76x3a4xs3lmNpvQjn5R0iIWAG6/HTZt2n7bpk1hu4jUTOYezYCUrKwsz87OjuTY6aBWrVAz35EZFBSkPh4RSQ0zm+HuWSU9ppmi1dQBB5Rvu4ikPyX0aur++6FBg+23NWgQtotIzaSEXk0NGACjRkHr1qGZpXXrcH/AgKgjE5GoxDOxSKqoAQOUwEVkG9XQRUTShBK6iEiaUEIXEUkTSugiImlCCV1EJE0ooYuIpAkldBGRNKGELiKSJpTQRUTShBK6iEiaUEIXEUkTSugiImlCCV1EJE0ooUul6WLVIlWDls+VSim8WHXh9U0LL1YNWtpXJNVUQ5dK0cWqRaoOJXSplG+/Ld92EUkeJXSpFF2sWqTqUEKXStHFqkWqDiV0qRRdrFqk6tAoF6k0XaxapGqIq4ZuZr3NbKGZfWVmQ0t4fIiZzTezuWb2vpm1TnyoIiJSmjITupnVBkYCpwJtgf5m1naH3WYBWe7eERgPPJToQEVEpHTx1NC7AF+5+2J33wKMBc4svoO7T3b3wtHInwAtExumiIiUJZ6E3gJYVux+TmzbrlwGvF3SA2Y22MyyzSw7Nzc3/ihFRKRM8SR0K2Gbl7ij2UAgC3i4pMfdfZS7Z7l7VvPmzeOPUkREyhTPKJccoFWx+y2B5TvuZGYnArcDPdz958SEJyIi8Yqnhj4dOMTM2pjZbkA/YELxHcysM/AU0NfdVyU+TBERKUuZCd3d84BrgInAAmCcu88zs3vNrG9st4eBhsArZjbbzCbsojgREUmSuMahu/tb7v5Ldz/I3e+PbbvL3SfEbp/o7r9w906xv76llyiSeFqXXWo6zRSVtKB12UW0loukCa3LLqKELmlC67KLKKFLmtC67FWT+jVSSwld0oLWZa96Cvs1li4F9239GkrqyaOELmlB67JXPerXSD1zL3EWf9JlZWV5dnZ2JMcWkeSrVSvUzHdkBgUFqY8nXZjZDHfPKukx1dBFJCnUr5F6SugikhTq10g9JXQRSQr1a6SeZoqKSNLoerOppRq6iEiaUEIXEUkTSugiImlCCV1EJE0ooYuIpAkldBGRNKGELiKSJpTQRRJIy8VWTTXlfdHEIpEE0WXwqqaa9L5otUWRBMnICMliR61bw5IlqY5GCqXb+6LVFkVSQJfBq5pq0vuihC6SIFoutmqqSe+LErpIgmi52KqpJr0vcSV0M+ttZgvN7CszG1rC48eZ2UwzyzOzcxMfpkjVp+Viq6aa9L6U2SlqZrWBL4GTgBxgOtDf3ecX2ycD2BO4CZjg7uPLOrA6RUVEyq+0TtF4hi12Ab5y98WxwsYCZwJFCd3dl8Qe05UCRUQiEk+TSwtgWbH7ObFt5WZmg80s28yyc3NzK1KEiIjsQjwJ3UrYVqHB6+4+yt2z3D2refPmFSlCRER2IZ6EngO0Kna/JbA8OeGIiEhFxZPQpwOHmFkbM9sN6AdMSG5YIlIZNWXtEtlemQnd3fOAa4CJwAJgnLvPM7N7zawvgJkdZWY5wHnAU2Y2L5lBi8iuFa5dsnQpuG9bu0RJPf1pLReRNJNua5fI9rSWi0gNUpPWLpHtKaGLpJmatHaJbE8JXSTN1KS1S2R7SugiaaYmrV0i29MVi0TS0IABSuA1kWroIiJpQgldRCRNKKGLiKQJJXQRkTShhC4ikiaU0EVE0oQSuohImlBCFxFJkWQva6yJRSIiKVC4rPGmTeF+4bLGkLhJYKqhi4ikwO23b0vmhTZtCtsTRQldRCQFUrGssRK6iEgKpGJZYyV0EZEUSMWyxkroIiIpkIpljTXKRUQkRZK9rLFq6CIiaUIJXUQkTSihi4ikCSV0EZE0oYQuIpImzN2jObBZLrA0koMnTjNgddRBVCE6H9voXGxP52N7lTkfrd29eUkPRJbQ04GZZbt7VtRxVBU6H9voXGxP52N7yTofanIREUkTSugiImlCCb1yRkUdQBWj87GNzsX2dD62l5TzoTZ0EZE0oRq6iEiaUEIXEUkTSugVYGatzGyymS0ws3lmdl3UMUXNzGqb2SwzezPqWKJmZk3MbLyZfRH7P3JM1DFFycxuiH1OPjezl82sftQxpYqZ/c3MVpnZ58W27W1m/2dmi2L/7pWo4ymhV0wecKO7Hw50Ba42s7YRxxS164AFUQdRRTwGvOPuhwFHUIPPi5m1AH4PZLl7e6A20C/aqFLqeaD3DtuGAu+7+yHA+7H7CaGEXgHuvsLdZ8ZubyB8YFtEG1V0zKwlcDrwTNSxRM3M9gSOA54FcPct7r4u2qgiVwfY3czqAA2A5RHHkzLuPgX47w6bzwReiN1+ATgrUcdTQq8kM8sAOgOfRhtJpIYDtwAFUQdSBRwI5ALPxZqgnjGzPaIOKiru/h0wDPgWWAGsd/d3o40qcr9w9xUQKofAPokqWAm9EsysIfAqcL27/xB1PFEwszOAVe4+I+pYqog6QCbwpLt3Bn4kgT+pq5tY+/CZQBtgf2APMxsYbVTpSwm9gsysLiGZj3H3f0QdT4S6AX3NbAkwFjjezEZHG1KkcoAcdy/8xTaekOBrqhOBb9w91923Av8AfhVxTFFbaWb7AcT+XZWogpXQK8DMjNBGusDdH4k6nii5+x/cvaW7ZxA6uya5e42tgbn798AyMzs0tukEYH6EIUXtW6CrmTWIfW5OoAZ3EsdMAC6K3b4IeCNRBesi0RXTDfgN8JmZzY5tu83d34owJqk6rgXGmNluwGLgkojjiYy7f2pm44GZhNFhs6hBywCY2ctAT6CZmeUAdwMPAuPM7DLCF955CTuepv6LiKQHNbmIiKQJJXQRkTShhC4ikiaU0EVE0oQSuohImlBCFxFJE0roIiJp4v8Dsc1nhPc7sKQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "epochs = range(1, len(acc) + 1)\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "plt.figure()\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a reminder, in chapter 3, the first naive approach to this dataset got you to a test\n",
    "accuracy of 88%. Unfortunately, this small recurrent network doesn’t perform well\n",
    "compared to this baseline (only 85% validation accuracy). Part of the problem is that\n",
    "your inputs only consider the first 500 words, rather than full sequences—hence, the\n",
    "RNN has access to less information than the earlier baseline model. The remainder of\n",
    "the problem is that SimpleRNN isn’t good at processing long sequences, such as text.\n",
    "Other types of recurrent layers perform much better. Let’s look at some moreadvanced layers. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_gpu",
   "language": "python",
   "name": "conda_gpu"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
